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ABSTRACT 


The charter school sector in the United States has grown steadily since the first charter 
school opened in 1992. As of the 2015-2016 school year, more than 6,800 charter schools served 
nearly 3 million students in forty states and the District of Columbia. Overall, research suggests 
that the average charter school performs about the same as nearby traditional public schools, but 
there is great variation in the effects of charter schools. Some charter schools are successful in 
boosting student achievement and others are not, which raises the question of what 
characteristics distinguish good charter schools from bad. This paper addresses this issue by 
summarizing the research on factors associated with successful charter schools. The research 
suggests that urban charter schools and charter schools primarily serving low-achieving and low- 
income students have the strongest positive impacts on student achievement. The policies most 
consistently found to be associated with positive charter school impacts include long school days 
or years, comprehensive behavioral policies with rewards and sanctions, and a mission that 
prioritizes boosting student achievement. In addition, moderately strong evidence suggests that 
high-dosage tutoring, frequent feedback and coaching for teachers, and policies promoting the 
use of data to guide teachers’ instructional practices are positively associated with charter 
schools’ achievement impacts. 
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I. INTRODUCTION 


Charter schools are a central component of current efforts to reform the public education 
system in the United States. These schools are publicly financed but free of many of the 
regulations that govern traditional public schools, such as those involving staffing, curriculum, 
and budget decisions. The first charter schools opened in Minnesota in 1992, and the charter 
school sector has grown steadily since that time. As of the 2015-2016 school year, more than 
6,800 charter schools served nearly 3 million students in forty states and the District of 
Columbia. 1 The growth in this sector has been particularly rapid in recent years; for example, 
there were just 2,800 charter schools serving 0.7 million students as of 2003. 

One of the original hopes for charter schools was that they would serve as a sort of 
educational laboratory. Without the oversight and regulations of traditional public school 
districts, charter schools would be free to innovate and test out new ideas. Some charter schools 
would fail, but the hope was that others would succeed and the new ideas and approaches that 
they successfully tested out could be used in other schools. In a May 2014 proclamation of 
National Charter Schools Week, President Barack Obama highlighted this point, saying that 
charter schools “that are successful can provide effective approaches for the broader public 
education system.” 

Today, more than two decades after the first charter schools opened, many good research 
studies have examined the effects of charter schools on student learning and achievement. Some 
have taken advantage of the fact that popular charter schools typically conduct admissions 
lotteries to admit students, allowing a comparison of subsequent outcomes among lottery 
participants who were and were not admitted by lottery. This experimental or quasi-experimental 
approach provides strong evidence on charter school effects but can only be applied to those 
charter schools that hold lotteries. Other studies have used rigorous non-experimental methods to 
produce evidence of the effects of broader groups of charter schools. 

Overall, the research suggests that the average charter school performs about as well, but no 
better, than nearby traditional public schools in terms of boosting student achievement. Among 
non-experimental studies examining broad sets of charter schools, for example, CREDO (2013a) 
found that average student achievement among charter school students in 27 states was no more 
than 0.01 standard deviations different from that of similar traditional public school students. 
Zimmer et al. (2012), examining charter schools in seven states, found that in most cases there 
were no statistically significant charter school effects. Studies using lottery-based methods to 
study broad sets of charter schools have had similar findings. Gleason et al. (2010) examined 36 
charter schools in 15 states and found no significant achievement effects on average. Using both 
lottery-based and non-experimental methods, Furgeson et al. (2012) studied charter schools in 22 
charter management organizations (CMOs) around the country and found no significant impacts 
on average. 

Although the typical charter school looks a lot like the typical traditional public school in 
terms of its effects on student achievement, another key finding from the research literature is 


1 Center for Education Reform. “Choice and Charter Schools: Facts.” Available at 
[https://www.edreform.com/issues/choice-charter-schools/facts/]. Accessed July 2016. 
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that there is great variation in the effects of charter schools. Some charter schools are successful 
in boosting student achievement and others are not. CREDO (2013a) found that many charter 
schools (25 to 29 percent) had positive effects and many (19 to 31 percent) had negative effects 
on students’ math and reading achievement. The remainder of charter schools did not 
significantly influence student achievement. In their lottery-based study, Gleason et al. (2010) 
found that impacts on reading achievement ranged from -0.43 to +0.33 standard deviations, with 
even greater variation in math impacts. Furgeson et al. (2012) found that among the 22 CMOs 
they studied, 1 1 had significant positive effects and 7 had significant negative effects in math. 2 

This variation in the effects of charter schools raises the question, what characteristics 
distinguish good charter schools from bad? Under what conditions are charter schools most 
likely to be successful? Can we identify the policies and practices that make a charter school 
successful? Can these policies and practices of successful charter schools be replicated in other 
schools — charter or traditional public schools — equally successfully? 

This paper addresses these questions by summarizing the research on factors associated with 
successful charter schools. The first section presents a framework for understanding how 
different factors might influence the success of a given charter school. Section II introduces the 
studies that have explored this issue systematically and summarizes their methodology for doing 
so. Section III summarizes the findings of these studies, and the final section discusses their 
implications. 

II. ANALYZING WHAT MAKES A CHARTER SCHOOL SUCCESSFUL: 

FRAMEWORK 


Defining school success 

In principle, a successful charter school should lead to positive short- and long-term student 
outcomes. In the short run students should master the important material covered by a school, 
and in the long run should do well throughout their educational careers and become productive 
and happy adults. Because prior research studies have not assessed school perfonnance based on 
all of these outcomes, the definition of school success used here is more limited, in two respects. 
First, studies have typically focused on the short-run outcome of student achievement as 
measured by performance on standardized assessments in reading and math. Second, charter 
school success is measured in relative terms, based on a comparison of charter schools with an 
alternative set of schools (usually nearby traditional public schools) that their students might 
otherwise have attended. 

Thus, a successful charter school is one in which student test scores are better than they 
would have been in this alternative, or counterfactual, set of schools. In practice, researchers 
compare test scores among charter school students with scores among a comparable set of 
students in nearby traditional public schools. 3 The traditional public school students used as a 


2 

" One might question whether there is similar variation in the impacts of individual traditional public schools. 
Zimmer et al. (2012) examined this issue and found significantly greater variation in school-level impacts in the 
charter school sector than in the traditional public school sector in most of the states they examined. 

One might broaden the set of counterfactual schools to include schools outside traditional public school systems. 
For example, students who apply to charter school but who fail to be admitted via lottery sometimes attend private 
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comparison or control group depend on the study’s design and methodology. Lottery-based 
studies follow lottery losers to whatever schools they attended in the years following the lottery. 
Many non-experimental studies select students with similar characteristics and baseline 
achievement levels as the charter school students from among traditional public school students 
in the same district as the charter school. 

This definition of what makes a charter school successful has implications for research on 
factors that lead to charter school success. Based on this definition, what makes a charter school 
successful depends not only on what is happening at that school, but also on what is happening at 
the nearby traditional public schools. 

The three Cs 

In the world of program evaluation, Weiss et al. (2014) formalized the analysis of sources of 
variation in program impacts — that is, what explains why some programs have more positive 
impacts than others. They summarize their analysis with reference to the three Cs: treatment 
contrast, clients, and context. Treatment contrast refers to the difference in services or resources 
received by treatment group members who participate in the program being evaluated and 
control group members who do not. It is this difference rather than simply the quantity and 
quality of the services or resources provided by the program that mediates the program’s 
impacts. The program’s clients are the individuals served by the program and who make up the 
evaluation’s treatment group. Characteristics of clients may influence, or moderate, the 
program’s impacts, as some types of clients may be helped by the program while others are not 
helped or are hurt. The context in which the program operates is another possible moderator. For 
example, some programs may work better during good economic times than during times when 
the economy is struggling (or vice versa). 

Although charter schools are not well-defined programs that are being evaluated, the 
framework developed by Weiss et al. (2014) can be used for understanding factors that may be 
related to charter school success. Here, the treatment contrast represents differences in the 
experiences of students attending charter schools versus those of students attending the schools 
serving as the counterfactual comparison group. These student experiences are captured using 
measures of school characteristics, policies, and practices, such as: 

• Amount of instruction received (e.g., length of the school day) 

• Curriculum and instructional approach 

• Approach to regulating student behavior 

• Quality of teachers and school leaders 

• Characteristics of student peers in the school 

Ideally, studies examining factors related to charter school impacts would measure the 
treatment contrast using differences in these characteristics between the charter schools being 
evaluated and the schools that control or comparison group students attend. In practice, most 


schools (Gleason et al. 2010). In practice, however, nearly all studies estimating impacts of charter schools on 
student achievement limit the analysis to students in charter schools or traditional public schools. 
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studies only have detailed infonnation on the charter schools’ characteristics, policies, and 
practices, and not on those of the nearby traditional public schools . 4 Rather than measuring the 
treatment contrast, these studies employ variables measuring the characteristics and practices of 
the charter schools alone. This approach implies that these studies cannot explain variation in 
charter school impacts arising from the fact that some are surrounded by lower quality traditional 
public schools than others . 5 However, they can explain variation in impacts associated with 
characteristics and practices of the charter schools themselves. 

A charter school’s clients are the students who attend the school. Different kinds of students 
may respond differently to charter schools, so these schools may be most successful for a 
particular kind of student. For example, students who are low achieving at the time they enter 
charter schools may benefit particularly from the typical charter school. If so, one would expect 
to see larger impacts for charter schools serving larger proportions of low-achieving students. 

Finally, the context in which the charter school operates may moderate their impacts. The 
neighborhoods in which charter schools are located may be related to their impacts, perhaps 
because the neighborhood influences the school’s ability to attract good teachers or students’ 
ability to focus on their schoolwork during nonschool hours. More generally, charter schools in 
urban settings may have different impacts than those in nonurban settings because of the 
characteristics of traditional public school districts in these areas . 6 

III. METHODS FOR IDENTIFYING FACTORS RELATED TO CHARTER SCHOOL 

SUCCESS 


The ideal study design 

To really understand how particular policies or practices influence a charter school’s 
success, one would ideally randomly assign these policies and practices to a group of charter 
schools and then measure their influence on the schools’ impacts. This design is not easy to 
implement, however, and has not been used within the charter school sector. 

This type of experimental approach has been conducted within the traditional public schools 
sector, with one set of schools randomly assigned to implement a particular practice or set of 
practices and other schools randomly assigned to a control group that would not implement the 


4 There are three exceptions. Gleason et al. (2010) and Furgeson et al. (2012) measured a treatment contrast for 
selected school characteristics using charter school and traditional public school differences based on data from a 
principal survey. Berends et al. (2010) used a different methodology and did not directly measure a treatment 
contrast but did account for the characteristics of both charter schools and traditional public schools based on both 
principal and teacher surveys. 

5 On the other hand, several of these studies focus on charter schools within a limited geographic area, such as a 
single district. To the extent that charter schools within a single district draw students away from the same 
traditional public schools or schools of similar quality, it is less likely that differences in charter school impacts will 
be driven by the characteristics of schools the control or comparison group students attend. 

6 Conceptually, the characteristics of the relevant set of traditional public schools should be reflected in measures of 
the treatment contrast. As described above, however, it is difficult to accurately measure these characteristics and, as 
a result, a contextual variable such as an indicator for being located in an urban area may serve as a proxy for 
traditional public school characteristics. 
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practice or practices. To cite just a few recent examples, this approach has been used to evaluate 
the effects of schoolwide implementation of data-driven instruction (Carlson et al. 2011; Slavin 
et al. 2013; Konstantopoulos et al. 2013), a schoolwide positive behavior supports approach to 
managing student behavior (Horner et al. 2009; Bradshaw et al. 2010); professional development 
for teachers (Garet et al. 2011); and instructional or curricular interventions in reading 
(Drummond et al. 2011) and math (Agodini et al. 2013). In addition, Fryer et al. (2014) 
randomly assigned a set of traditional public schools to implement a set of practices, including 
extensive teacher feedback, data-driven instruction, high-dosage tutoring, and expanded 
instructional time. 7 

Randomly assigning practices to schools is particularly challenging to implement in the 
charter school sector, however. One of the essential features of charter schools is their 
operational autonomy, which allows leaders to implement the educational approach that they 
believe is most likely to be successful given their students. Randomly assigning them a particular 
practice would work against this operational autonomy. In addition, charter schools typically 
implement a bundle of policies and practices, which define their identities. For example, No 
Excuses charter schools emphasize a particular approach to maintaining student discipline, high 
expectations for academic achievement, a focus on traditional reading and math skills, increased 
instructional time, and selective teacher hiring (Carter 2000; Thernstrom and Themstrom 2003). 
Imposing a single policy or practice into this mix may have little effect or be counterproductive. 

Observational studies 

Studies that have addressed the question of what factors may be responsible for the success 
of certain charter schools have used a non-experimental or observational approach. This 
approach involves observing how policies and practices actually implemented by charter schools 
are correlated with measures of their success, which are typically the estimated impacts of the 
school on student achievement. Most studies of the factors related to charter school success use 
either a lottery-based or strong non-experimental design to rigorously estimate charter school 
impacts. To measure charter schools’ characteristics, policies, and practices in some detail, the 
studies typically use surveys of school leaders or staff. 

Once the studies have estimated impacts and measured the policies and practices of a sample 
of charter schools, they estimate the correlation between factors thought to influence charter 
school success and the schools’ impacts on student achievement. In some cases, this correlational 
analysis is bivariate, where the basic question being addressed is whether a set of charter schools 
that have a particular feature (Feature A) have, on average, more positive (or more negative) 
impacts than charter schools without Feature A. In other cases the analysis is multivariate, with 
statistical controls added to the analysis for other relevant features of the school. A multivariate 
correlational analysis would address the question of whether charter schools with Feature A have 
more positive or negative impacts than charter schools without this feature, controlling for these 
other features. The idea behind this analysis is that schools with Feature A may be more 
successful (have more positive impacts) not because of A but because they are also more likely 
to have one of these other features. 


7 Because this study defined these practices on the basis of their use in successful charter schools, it is described in 
greater detail in Section IV. 
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Limitations of observational studies 

The methodology for determining what factors may explain the success of certain charter 
schools has several limitations. The most important limitation is that correlation is not the same 
thing as causation. In observational studies, policies and practices correlated with positive charter 
school impacts may be correlated with other, unobserved characteristics, policies, and practices 
that are actually driving the observed success. Without random assignment, one cannot be certain 
that the factors included in the analysis are causally related to charter school impacts. Suppose 
that charter schools with extended instructional time have more positive impacts, even after 
controlling for several other factors. One possibility is that this correlation is causal — that having 
extended hours leads to charter school success. An equally plausible possibility, however, is that 
charter schools with extended hours also tend to have other characteristics or follow other 
practices that lead to their success. These schools might have a particularly effective principal or 
dedicated teachers, for example. Or they may happen to be located in areas with especially low- 
quality traditional public schools. Understanding whether the correlation between extended hours 
and impacts is causal has important policy implications, because if the correlation is spurious 
rather than causal, implementing a policy of lengthening the school day more widely will not 
lead to more positive student outcomes. 

Another limitation of observational studies of factors related to charter school impacts is that 
they have relied on small samples of charter schools (because estimating impacts and collecting 
data on school characteristics, policies, and practices can be challenging and costly). The typical 
study has analyzed factors related to impacts using a sample of about 30 charter schools. Thus, 
the statistical power of these analyses is low, in that a relationship between a particular factor 
and charter school impacts would have to be quite strong to be consistently found to be 
statistically significant. This problem is made worse by the fact that the number of factors that 
could potentially influence a charter school’s success is large. So although the typical study has a 
sample of about 30 to 40 charter schools to analyze, the number of different characteristics, 
policies, or practices that could conceivably influence the schools’ impacts on student 
achievement greatly exceeds 30 to 40. 8 

Finally, many factors one might expect to be the most important contributors to a charter 
school’s ability to boost student achievement relative to nearby traditional public schools are 
difficult to observe or measure and are not included in these analyses. As previously mentioned, 
a charter school’s impacts depend in part on the characteristics and quality of the traditional 
public schools to which they are compared, which are difficult to measure and often excluded. 
Similarly, some key charter school characteristics are difficult to measure and typically excluded 
from these analyses. For example, a charter school’s success depends on the effectiveness of its 
staff, especially the principal and teachers. But principal and teacher quality is difficult to 
measure objectively. Studies have tried to find proxies for quality using measures such as 
certification, experience, and educational attainment, but these objective measures have been 
shown to be weak proxies for true teacher and principal quality (Rivkin et al. 2005; Aaronson et 
al. 2007). 


o 

The studies reviewed here have explicitly tried to limit the number of factors they’ve examined given the limited 
degrees of freedom in analysis based on a sample of 30 to 40, but still have examined correlations with charter 
school impacts of between 1 1 and 43 different factors. 
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IV. FACTORS ASSOCIATED WITH CHARTER SCHOOL SUCCESS 


Description of studies 

The findings presented here are from studies that set out explicitly to examine a wide range 
of factors that might explain charter school success. Other studies have presented evidence on 
how one or two individual factors are related to charter school impacts, but they did not aim to 
systematically explain factors related to charter school success. Those studies are not reviewed 
here. 

A brief description of each study that has systematically explored factors that may explain 
charter school success is presented below. Table 1 shows additional details about these studies. 

• Hoxby et al. (2009): This study used a lottery-based design to estimate impacts among a 
sample of 32 charter schools in New York City. Based on data on the characteristics of these 
charter schools, the authors conducted bivariate and multivariate analysis of the 
relationships between these charter school characteristics and their impacts on student 
achievement. 

• Berends et al. (2010): This study used a non-experimental design to estimate average 
impacts among a national sample of 76 charter schools. Data on school characteristics was 
obtained from teacher and principal surveys administered at both the charter schools and a 
matched set of traditional public schools. Unlike the other studies reviewed here, this study 
did not estimate separate impacts for each charter school in the analysis and examine 
correlations between school characteristics and impacts. Rather, it used a multivariate 
regression model to examine the relationship between individual school characteristics and 
student achievement. In other words, the analysis used variation in school characteristics 
within both the charter and traditional public school sectors to estimate the relationship 
between these characteristics and student achievement. 

• Gleason et al. (2010): This study used a lottery-based design to estimate impacts among a 
sample of 36 charter schools in 15 states. Using data on school characteristics in both these 
charter schools and the traditional public schools that lottery losers attended, the authors 
examined the correlations between school characteristics — contrasting charter and 
traditional public schools when possible — and impacts on student achievement. 

• Furgeson et al. (2012): This study used both a lottery-based design and a non-experimental 
design to estimate impacts among a sample of 22 charter management organizations 
(CMOs) with 68 schools in 8 states. The analysis of factors related to CMO impacts were 
based on the non-experimental impact estimates. Unlike the other studies reviewed here, this 
study analyzed impacts at the CMO level, rather than the individual charter school level. 
Using data on both the CMO schools and nearby traditional public schools, the authors 
examined the correlations between CMO characteristics — contrasting charter and traditional 
public schools when possible — and impacts on student achievement. 
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Table 1. Summary of studies systematically examining factors related to charter school impacts 


Study 

Location 

Years 

Number of 

charter 

schools 

Number of 

factors 

examined 

Design for 
estimating 
impacts 

Data source for measuring 
school characteristics and 

other factors related to charter 
school impacts 

Design for estimating 
relationship between 
factors and charter school 
impacts 

Hoxby et al. 

(2009) 

New York City 

2000-2001 

through 

2007-2008 

33 

28 

Lottery-based 

Charter schools 
administrators, NYC Dept, of 
Education (measured for 
charter schools only) 

School-level bivariate and 
multivariate analysis 3 

Berends et al. 

2010 

National 

2002-2003 

through 

2005-2006 

76 

12 

Non- 

experimental 

Teacher and principal surveys 
(charter schools and 
traditional public schools) 

Student-level regression 
model b 

Gleason et al. 
(2010) 

1 5 states 

2004-2005 

through 

2007-2008 

36 

18 

Lottery-based 

Principal survey (charter 
schools and traditional public 
schools) 

School-level bivariate and 
multivariate analysisa 

Furgeson et al. 
(2012) 

8 states 

2002-2003 

through 

2009-2010 

22 CMOS, 
68 schools 

6 (43) c 

Non- 

experimental / 
lottery-based d 

Principal survey (charter 
schools and traditional public 
schools) and teacher survey 
(charter schools only) 

School-level bivariate and 
multivariate analysis 3 

Angrist et al. 

(2013) 

Massachusetts 

2001-2002 

through 

2010-2011 

33 

1 1 (42) e 

Non- 

experimental / 
lottery-based 11 

School administrator survey 
(charter schools only) 

School-level bivariate and 
multivariate analysis 3 

Dobbie and Fryer 
(2013) 

New York City 

2003-2004 

through 

2010-2011 

29 

CO 

o 

Non- 

experimental / 
lottery-based" 1 

Principal, teacher, and student 
surveys and video 
observations (charter schools 
only) 

School-level bivariate and 
multivariate analysis 3 

Tuttle et al. (2013) 

1 3 states 

2003-2004 

through 

2010-2011 

36 

17 

Non- 

experimental / 
lottery-based" 1 

Principal survey (charter 
schools only) 

School-level bivariate and 
multivariate analysis 3 


a The bivariate analysis involved regressing a measure of charter school impacts on a single characteristic of charter schools or examining this relationship 
between school characteristics and impacts in some similar way. The multivariate analysis involved regressing impacts on several school characteristics 
simultaneously. 

b Berends et al. (2010) estimated a regression model in which student achievement was regressed on the student’s prior achievement level, demographic 
characteristics, and characteristics of the student’s school (regardless of whether or not the school was a charter school). One of the school characteristics 
included in the regression was charter school status. Thus, this study used variation in school characteristics across schools in the charter and traditional public 
school sectors to capture the relationship between school characteristics and student achievement. 

c Furgeson et al. (2012) conducted the primary, confirmatory analysis using 6 factors, but conducted exploratory analysis with a total of 43 factors. 

d Non-experimental estimates were the primary estimates used in estimating the relationship between charter school characteristics and impacts, though as part of 
the larger study impacts were estimated using lottery-based methods as well. 

e Angrist et al. (2013) conducted the primary, confirmatory analysis using 1 1 factors but conducted exploratory analysis with a total of 42 factors. 
f Dobbie and Fryer (2013) conducted the primary, confirmatory analysis using 9 factors but conducted exploratory analysis with a total of 40 factors. 

CMO = charter management organization. 
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• Angrist et al. (2013): This study used both a lottery-based design and a non-experimental 
design to estimate impacts among a sample of 33 Massachusetts charter schools. The 
analysis of factors related to charter school impacts was based on the non-experimental 
impact estimates. Using data on the characteristics of these charter schools, the authors 
examined the correlations between school characteristics and impacts on student 
achievement. 

• Dobbie and Fryer (2013): This study used both a lottery-based design and a non- 
experimental design to estimate impacts among a sample of 29 New York City charter 
schools. The analysis of factors related to charter school impacts was based on the non- 
experimental impact estimates. Using data on the characteristics of these charter schools, the 
authors examined the correlations between school characteristics and impacts on student 
achievement. 

• Tuttle et al. (2013): This study used both a lottery-based design and a non-experimental 
design to estimate impacts among a sample of 36 KIPP (Knowledge Is Power Program) 
schools. The analysis of factors related to charter school impacts was based on the non- 
experimental impact estimates. This study differed from the others examined here in that the 
sample of charter schools was restricted to those from a single CMO. Using data on the 
characteristics of these KIPP schools, the authors examined the correlations between school 
characteristics and impacts on student achievement. 

Key findings — context and student characteristics 

These studies examined different charter school samples and focused on somewhat different 
sets of school characteristics, practices, and policies, but a number of common themes can be 
seen in the conclusions reached in these studies about what factors were associated with the most 
successful charter schools. Among studies that examined the context in which charter schools 
operate and the characteristics of their students, two key findings emerged: 

• The most successful charter schools are those located in urban areas 

Just two of these studies directly examined the relationship between a charter school’s urban 
location and its achievement impacts, but each study found a strong relationship between 
urban location and more positive impacts. Angrist et al. (2013) found that among 
Massachusetts middle schools, a year at an urban charter school compared with nearby 
traditional public schools led to a statistically significant increase of 0.15 standard deviations 
in English/language arts achievement by students, whereas the impact of nonurban charter 
schools was negative and significant (-0.14). The difference between urban and nonurban 
charter schools impacts in math was even larger. Among a sample of charter schools in 
multiple states, Gleason et al. (2010) found that after two years the impact of being admitted 
to an urban charter school on student math achievement was positive and significant (0.16) 
while the impact of nonurban charter schools was negative and significant (0.14). 

Some studies have not examined urban-nonurban differences because they focus entirely on 
urban charter schools. However, the overall pattern of results across studies supports the 
finding of more positive impacts in urban charter schools. Studies focusing entirely on urban 
charter schools have generally found positive impacts, including charter school studies in 
Boston (Abdulkadiroglu et al. 2013), Chicago (Hoxby and Rockoff 2005), and Los Angeles 
(CREDO 2014), and New York City (Hoxby et al. 2009; Dobbie and Fryer 2013; CREDO 
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2013b). By contrast, broader studies that include urban and nonurban charter schools have 
found impacts that are close to zero and not statistically significant (Bifulco and Ladd 2006; 
Sass 2006; Gleason et al. 2010; Furgeson et al. 2012; Zimmer et al. 2012; CREDO 2013a). 

• Charter schools serving low-achieving and low-income students tend to have more 
positive impacts than those serving higher achieving students and those from more 
advantaged backgrounds 

Few studies examined whether charter schools’ impacts depended on the composition of 
their students, though several studies examined impact differences for students with 
particular characteristics. However, Gleason et al. (2010) found a strong association between 
the percentage of a charter school’s students who are low-achieving and low-income and the 
school’s impacts. Charter schools serving a large proportion of low-income students had 
significant positive impacts in math achievement (0.18 standard deviations); those serving 
fewer low-income students had significant negative impacts (-0.24). A similar pattern 
existed between the proportion of low-achieving students (as measured at the time of 
enrollment) and a charter school’s impacts. 

As students in urban areas tend to have lower incomes and lower achievement levels than 
students outside urban areas, these two key findings above reflect the same trends. Angrist et 
al. (2013) explored whether the advantage of urban over nonurban charter schools could be 
explained by the fact that urban schools served larger proportions of students for whom 
charter school impacts tend to be more positive. They found that these student 
characteristics explained a portion of the urban charter school advantage but not the entire 
difference. Thus, urban charter schools must have more positive impacts either because their 
policies and practices are more successful, or because the policies and practices of the 
traditional public schools they are competing against are particularly unsuccessful. 

Key findings — school policies and practices consistently associated with 
positive charter school impacts. 

Several charter school policies or practices were positively related to impacts on student 
achievement consistently across a number of different studies. 

• The most successful charter schools consistently enforce a comprehensive behavior 
policy with rewards for positive behavior and sanctions for negative behavior 

Although different studies used different measures of schools’ behavior policies, several 
found that charter schools that had high expectations for student behavior and strictly 
enforced their rules governing behavior had more positive impacts on student achievement: 

Furgeson et al. (2012) found that “comprehensive behavior policies” were positively 
associated with student impacts. This behavior measure was based on whether the school 
used “consistent behavior standards and disciplinary policies,” zero-tolerance policies 
for potentially dangerous behaviors, rewards and sanctions, and parent or student 
responsibility agreements. 

Hoxby et al. (2009) found more positive impacts among schools with disciplinary 
policies “based on the idea that expecting small courtesies and punishing small 
infractions ... is important.” The authors contrasted this approach with disciplinary 
strategies that focused mainly on preventing large infractions. 
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Dobbie and Fryer (2013) found more positive impacts among schools with higher values 
on a 10-item index aimed at capturing the extent to which schools had a “strict 
disciplinary policy.” 

Angrist et al. (2013) found that the extent to which the school emphasized “discipline 
and comportment” was positively correlated with impacts. They argued that this policy 
is an important part of schools’ No Excuses identity, which they also found to be 
strongly correlated with achievement impacts. 

Finally, several studies examined charter school networks known for an emphasis on 
high expectations for student behavior, and found positive impacts among these charter 
schools. Teh et al. (2010) found positive impacts among schools in the Achievement 
First and Uncommon Schools CMOs. KIPP schools have also been found to have strong 
positive impacts on student achievement (Angrist et al. 2012; Gleason et al. 2014). All 
KIPP schools use an approach that includes high expectations for student behavior, but 
Tuttle et al. (2013) found there is variation in disciplinary policies even within individual 
KIPP schools that is associated with impacts. KIPP schools with the most positive 
impacts had more positive values of an index capturing the extent to which behavior 
standards and discipline policies with rewards and sanctions are established and enforced 
consistently across the entire school. 

• Charter schools with longer school days, school years, or both tend to have more 
positive impacts on student achievement 

Nearly all studies reviewed here found a positive association between a charter school’s 
instructional time (length of the school day or year) and impacts on student achievement 
(Hoxby et al. 2009; Gleason et al. 2010; Furgeson et al. 2012; Dobbie and Fryer 2013; 
Angrist et al. 2013). However, several found that although this relationship was statistically 
significant in bivariate models, it was not significant in multivariate models (Gleason et al. 
2010; Furgeson et al. 2012; Angrist et al. 2013). When other policies and practices were 
controlled, charter schools with longer school days or years no longer had significantly more 
positive impacts. This pattern of results suggests that it may not be the additional time in 
school that leads to more positive impacts, but rather some other policies or practices 
associated with time in school. 9 

An interesting relationship between time in school and impacts was evident in KIPP schools 
(Tuttle et al. 2013). Overall, the average KIPP school day is nine hours, much longer than the 
average traditional public school day. So the positive impacts across the KIPP network 
(Gleason et al. 2014) support the notion that long school days could contribute to positive 
impacts. The school day among KIPP schools varies from about 8 to 10 hours, and Tuttle et 
al. (2013) found that those with the longest days had less positive impacts than those with 
shorter days. Upon further exploration, however, they found that what happened during the 
school day mattered. Schools that spent more time on core instructional activities had more 
positive impacts, whereas those that spent more time on activities other than core instruction 
had less positive impacts. In other words, they found that a long school day still was 
associated with more positive impacts so long as the time was spent on core instruction. 


9 An alternative possibility is that time is schools is so strongly correlated with these other policies and practices that 
it is not possible to disentangle their effects. 
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Research on extended time in traditional public schools also suggests that extra time may 
have beneficial effects only if it is devoted to instruction in core subjects. A non- 
experimental evaluation of the expanded learning time (ELT) initiative in Massachusetts, 
which added at least 300 hours to the school year but did not restrict the additional time to 
instructional activities, found that the additional time did not significantly affect student 
achievement (Checkoway et al. 2012). By contrast, Cortes et al. (2014) found that a double 
dose of algebra instruction had positive impacts on student achievement and educational 
attainment. 

• Charter schools that prioritize the objective of boosting students’ academic 
achievement above other possible educational objectives have more positive impacts on 
student test scores 

Several studies used measures designed to capture schools’ focus on improving their 
students’ academic achievement. All schools presumably want their students to perfonn at 
high levels, but these studies attempted to measure which schools placed more emphasis on 
this goal than on other goals. Berends et al. (2010) measured “focus on achievement” using 
teachers’ ratings on items such as the extent to which they expected students to complete 
assignments and the extent to which they set high expectations for students’ academic work. 
Hoxby et al. (2009) measured the extent to which the school’s mission statement 
emphasized academic perfonnance. Furgeson et al. (2012) used school principals’ reports on 
whether the school places high importance on students “exceeding state academic 
standards.” Dobbie and Fryer (2013) used principals’ reports on whether the school places a 
“relentless focus on academic goals and having students meet them,” noting that other 
potential priorities not selected by these schools involved “the social and emotional needs of 
the whole child” and “building a student’s self-esteem through positive reinforcement.” In 
all of these studies, the authors found a positive correlation between charter schools’ focus 
on academic achievement on their impacts on students’ math or English/language arts 
scores. 

Key findings — school policies and practices with moderate evidence of 

positive associations with charter school impacts 

Based on somewhat more limited evidence, the research has identified several additional 

factors potentially related to charter school impacts. 

• Some evidence suggests that charter schools providing frequent feedback and coaching 
to teachers have more positive impacts on student achievement 

Dobbie and Fryer (2013) and Furgeson et al. (2012) each measured how much coaching and 
feedback charter school teachers receive on their instructional practices, using indexes based 
on several different indicators. Each found that the most successful charter schools provided 
teachers with the most coaching and feedback, based on both the bivariate and multivariate 
analysis. Among KIPP schools, on the other hand, Tuttle et al. (2013) found no significantly 
relationship between KIPP impacts and a measure of teacher professional development and 
mentoring. Among traditional public schools, Glazennan et al. (2010) studied a teacher 
induction intervention that provided teachers with various forms of coaching and support. 
They found positive impacts of this intervention but only after teachers had been exposed to 
it for three years. 
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• Charter schools that promote the use of student data to guide teachers’ instructional 
practices may be more successful in boosting student achievement. 

A popular approach to helping teachers improve their instructional practices involves using 
data from interim student assessments to provide real-time information to teachers on 
students’ mastery of particular concepts (Hamilton et al. 2009). Several studies offered some 
support for the promise of this data-driven instruction approach in charter schools. Hoxby et 
al. (2009) found in bivariate analysis that charter schools that administer more internal 
diagnostic assessments per year had more positive impacts, though this relationship was not 
significant in multivariate analysis. Angrist et al. (2013) found that using infonnal tests to 
gauge understanding was positively correlated with impacts. Dobbie and Fryer (2013) found 
that an index of data-driven instruction was positively associated with impacts, although not 
significantly. Finally, Furgeson et al. (2012) found that although measures of a school’s 
efforts to get student data to teachers were not significantly associated with impacts on 
student achievement, a measure of how much teachers actually used data to help with their 
instruction was positively associated with impacts. In other words, it was not enough to 
simply get the student data to teachers, but they must actually have used the data to guide 
their practices. 

Evidence from traditional public schools has not shown a strong positive relationship 
between data-driven instruction and student achievement. Carlson et al. (2011), Slavin et al. 
(2013), and Konstantopoulos et al. (2013) all found either no significant effects of data- 
driven instruction on student achievement or small positive effects. Again, however, these 
studies examined interventions that focused largely on producing student data and getting it 
to teachers rather than on helping the teachers figure out how to use the data to improve their 
instruction. 

• Limited evidence suggests that high-dosage tutoring helps charter schools improve 
student achievement 

Dobbie and Fryer (2013) measured the extent to which charter schools offered students 
high-dosage tutoring or frequent instruction in very small groups — instruction in groups of 
six or fewer students four or more times a week. They found a positive association between 
tutoring and charter school impacts, particularly in English/language arts. Angrist et al. 
(2013) also found a positive relationship between a measure of tutoring for all students and 
charter school impacts in math. Other studies did not directly examine the relationship 
between the availability of tutoring in charter schools and the schools’ impacts. Research in 
traditional public schools is mixed, but a recent study of the provision of supplemental 
educational services such as tutoring and other academic services under the No Child Left 
Behind (NCLB) Act show no evidence of positive impacts on student test scores (Deke et al. 
2014). 

Key findings — school policies and practices not associated with charter 
school impacts 

Studies have also identified policies and practices that do not appear to be related to impacts. 
These findings do not necessarily mean that these policies and practices are not relevant or 
important, given the limitations of the studies’ research designs. Just as we cannot necessarily 
conclude that factors correlated with charter school impacts are responsible for these schools’ 
success, we cannot rule out the possibility that factors not significantly correlated with charter 
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schools’ impacts do contribute to their success. However, the evidence to date shows no 

significant relationships between the following factors and charter school impacts: 

• Charter school funding. Angrist et al. (2013) and Dobbie and Fryer (2013) failed to find a 
statistically significant relationship between charter schools’ per pupil expenditures and 
impacts. Gleason et al. (2010) found a significant bivariate relationship between per pupil 
revenues and impacts in math, but the relationship was no longer significant in the 
multivariate model and also was not significant in English/language arts. A big caveat with 
this research is that it is difficult to accurately measure charter school funding, given 
complexities of revenues and costs associated with school facilities as well as serving 
special needs students. Moreover, none of these studies accounted for the funding at nearby 
traditional public schools, which could presumably also influence impacts. 

• Class size. Several studies found no relationship between class size or student-teacher ratio 
and charter school impacts (Hoxby et al. 2009; Furgeson 2012; Dobbie and Fryer 2013; 
Tuttle et al. 2013). Gleason et al. (2010) found a significant relationship but in a 
counterintuitive direction, with impacts more positive in charter schools with a larger 
student-teacher ratio. The major piece of experimental evidence on the effects of class size 
in traditional public schools comes from Project STAR, which compared the perfonnance of 
students in small classes (13-17 students) with those in larger classes (22-25 students) in 
kindergarten through 3rd grade. This study showed evidence of positive effects of very 
small classes, especially in the early grades (Finn and Achilles 1990). 

• Teacher qualifications. Dobbie and Fryer (2013) found no significant positive association 
with charter school impacts of two measures of teacher qualifications, teacher certification 
and having a master’s degree. Similarly, Gleason et al. (2010) and Tuttle et al. (2013) find 
no significant association between the proportion of experienced teachers in a charter school 
and the school’s impacts. These findings are largely consistent with research in traditional 
public schools, which has found no consistent link between student learning and teacher 
qualifications, with the exception of a teacher’s relative newness to teaching (Rivkin et al. 
2005; Aaronson et al. 2007). 

• Charter management organization affiliation. Some individual CMOs have been found to 
have strong positive impacts on student achievement (Teh et al. 2010; Gleason et al. 2014), 
but studies have found that overall, there is no significant relationship between whether a 
charter school is affiliated with a CMO and its impacts (Hoxby et al. 2009; Gleason et al. 
2010). 10 In addition, studies that directly estimated impacts of CMO schools across multiple 
CMOs found that average impacts were not significantly different from zero (Furgeson 
2012; CREDO 2013c). 

• Policies promoting parent involvement. Some charter schools make more of an effort than 
others to involve parents in their students’ education. Dobbie and Fryer (2013) developed an 
index of parent engagement capturing feedback provided to parents on students’ behavior, 
academic performance, and other issues. Hoxby et al. (2009) and Angrist et al. (2013) 
measured whether charter schools made use of parent contracts. Tuttle et al. (2013) 


10 One study did find some evidence of a relationship between CMO status and charter school impacts. Dobbie and 
Fryer (2013) found that New York City charter schools affiliated with a CMO had significantly more positive 
impacts at the elementary school level, though differences at the middle school level were not significant. 
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developed an index of parent involvement. In each case, the researchers found no significant 
relationship between these measures of parent engagement or involvement and charter 
school impacts. 

• Charter school age. It is possible that charter schools may become more effective as they 
mature. However, Hoxby et al. (2009) and Gleason et al. (2010) found no significant 
relationship between the age of a charter school and its impacts on student achievement. One 
caveat to these findings is that both studies were lottery-based and focused on 
oversubscribed charter schools. Thus, very young charter schools, which would presumably 
be less likely to be oversubscribed, may have been underrepresented. However, CREDO 
(2013c) focused on a broader sample of charter schools and found that charter school 
perfonnance in later years can be predicted well from perfonnance in their early years. 

• Enrollment. Furgeson et al. (2012) found that total school enrollment was not significantly 
related to charter school impacts. Similarly, Berends et al. (2010) found no significant 
relationship between enrollment and student achievement. Gleason et al. (2010) found a 
significant negative relationship between higher school enrollment and charter school 
impacts — impacts were more positive is schools with lower enrollment. However, the 
relationship between enrollment per grade and impacts was not statistically significant, 
suggesting that the enrollment effect may have been related to the number of grades served. 

V. DISCUSSION: WHAT DO WE KNOW AND HOW CAN WE LEARN MORE? 


Research on the relationship between charter schools’ policies and practices and their 
impacts on student achievement is motivated by a desire to understand what explains the success 
of certain charter schools. The hope is that by understanding factors linked to their success, these 
policies and practices might be used successfully in other charter schools or traditional public 
schools. Charter school networks that employ these policies and practices might be expanded, or 
traditional public schools and new charter schools might be encouraged to use the same policies 
and practices. Ultimately, the hope is that the education system as a whole might be improved. 

What do we know? The research provides consistent evidence that some policies and 
practices are correlated with more positive charter school impacts on student achievement. The 
most successful charter schools tend to be those that consistently enforce a comprehensive 
behavior system, have long school days and or years, and put their highest priority on helping 
students meet high expectations for academic achievement. Successful charter schools tend to 
operate in urban areas and serve low-income and low-achieving students. More limited evidence 
suggests that successful charter schools give teachers frequent feedback and coaching, encourage 
and facilitate the use of student data to help teachers improve their practices, and provide high 
dosage tutoring to students. 11 On the other hand, a number of charter school characteristics, 
policies, and practices have been found to be uncorrelated with their impacts, including the 
school’s age, CMO affiliation, funding, enrollment, average class size, teacher qualifications, 
and the extent to which they encourage parent involvement. 


1 1 Some evidence suggests that successful charter schools tend to most or all of these policies and practices in 
clusters, rather than just one or two of them (Furgeson et al. 2012; Angrist et al. 2013; Dobbie and Fryer 2013). 
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The wisdom of encouraging the use of policies and practices that have been found to be 
correlated with charter school impacts depends in part on two characteristics of the underlying 
research. First, does the research have good internal validity? In other words, are these policies 
and practices actually responsible for the success of the schools included in these studies, or is 
the observed correlation spurious? Second, does the research have good external validity? Even 
if we conclude that certain policies and practices promote the success of the charter schools in 
these studies, would these relationships also hold in other samples of charter schools or in 
traditional public schools? 

The research in this area provides little guidance on questions of internal and external 
validity. As discussed previously, the limitations of the studies’ research designs — small samples 
and limited ability to control for confounding factors — give us reasons to be skeptical of their 
internal validity. The policies and practices found in successful charter schools may simply be 
correlated with other factors that are actually responsible for the schools’ positive impacts. For 
example, charter school impacts may be driven by a principal’s ability to identify and hire 
effective teachers, and such principals may also happen to work in schools with long school 
days. Because we cannot observe a principal’s ability to hire good teachers, we simply observe a 
positive correlation between the length of the school day and charter school impacts. However, if 
we were to open a new charter school with a long school day, that school would not necessarily 
be successful because it would not necessarily have a principal with the ability to identify and 
hire good teachers. 

The external validity of existing studies is questionable because they have focused on a 
limited set of charter schools with particular characteristics and operating in a particular context. 
The studies have relied heavily on both urban and oversubscribed charter schools. What is true 
for these schools may not be true for suburban or rural charter schools or for schools that are not 
oversubscribed. In addition, the studies have focused on charter schools operating at a time in 
which charter schools make up a relatively small proportion of all public schools in the United 
States. Would the policies and practices associated with charter school success in this context 
work equally well if charter schools served a larger proportion of students? In that situation, 
charter schools might have to adapt their practices to attract students or might struggle to find 
enough effective teachers. 

Believing that the policies and practices correlated with charter school impacts would work 
equally well in traditional public schools requires an even greater leap. Charter schools are 
schools of choice. Students do not have to attend charter schools, and the schools may have 
attributes that make them less attractive to certain types of students. Students who are especially 
unmotivated are unlikely to choose to attend a school with nine-hour school days, for example. 
Traditional public schools, by contrast, are less likely to be schools of choice and typically must 
serve all students living in a particular geographic area (unless these students choose to go 
elsewhere). It is difficult to know whether the policies and practices that are effective in boosting 
student achievement in a school of choice would be equally effective in a traditional public 
school serving all students who live in a given neighborhood. 

One relevant aspect of the context in which charter schools operate involves the 
characteristics of charter school students’ peers. If charter schools attract or retain only the most 
motivated or best behaved students in a neighborhood, then students attending these schools may 
benefit from positive peer effects. In a critique of KIPP, Kahlenberg (2011) argued that “the big 
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difference between KIPP and regular public schools ... is that whereas struggling students come 
and go at regular schools, at KIPP, students leave but very few new students enter. Having few 
new entering students is an enonnous advantage not only because low-scoring transfer students 
are kept out but also because in later grades, KIPP students are surrounded only by successful 
peers.” In the case of KIPP, research suggests that entering students have about the same prior 
test scores as traditional public school students and that patterns of attrition from KIPP are 
similar to those of nearby traditional public schools (Nichols-Barrer et al. 2013). However, KIPP 
schools are less likely than traditional public schools to replace low-achieving exiting students 
with new students who are equally low-achieving. So patterns of positive selection into KIPP 
based on prior achievement do emerge over time. Moreover, little is known about selection into 
KIPP (or charter schools more generally) based on student motivation or behavior. Thus, it is 
possible that students at charter schools like KIPP benefit from positive peer effects, and that 
these peer effects would be difficult to replicate at scale or in traditional public schools. 

Scaling up successful charter school networks brings additional challenges. For successful 
networks to remain successful as they grow larger, they must be able to maintain the policies and 
practices that made them successful. As they open new schools, can they continue to attract 
enough students and the right kind of students to fill their seats? Can they attract enough teachers 
and school leaders to be able to effectively implement these policies and practices? At KIPP, for 
example, the average principal reports working 74 hours per week (Tuttle et al. 2013). If the 
number of KIPP schools in a district doubles or triples, will the CMO continue to be able to 
attract principals willing to work that hard and who remain effective school leaders? Finally, as 
CMOs grow, they face increasing administrative burden, and their internal bureaucracy 
presumably must grow to meet this burden. Will this growing internal bureaucracy lead to 
unanticipated problems? 

How can we leam more? If policies and practices correlated with charter school impacts are 
not necessarily responsible for those impacts, what lessons can we draw from this research? 
Research examining factors related to charter school impacts should best be thought of as 
hypothesis-generating research. Although not providing conclusive evidence about what policies 
and practices bring school success, the research provides suggestive evidence about policies and 
practices that may help boost student achievement and that are worth further investigation. And 
the best strategy for learning more about the effectiveness of these promising policies and 
practices involves testing them out elsewhere and carefully measuring their effects. In particular, 
a set of policies and practices found to be correlated with charter school impacts can be 
purposefully implemented in new schools. This purposeful implementation will allow 
researchers to set up research designs that will provide more rigorous evidence about their effects 
on student achievement. 

In practice, such research studies could be implemented in two different ways. First, a 
package of policies and practices associated with successful charter schools might be randomly 
assigned to a group of schools that would agree to participate in a research study. After these 
policies and practices had been carefully implemented in the randomly selected schools, 
subsequent student achievement and other outcomes in these schools could be compared with 
those of students in schools that also participated in the study but were not randomly selected to 
implement the intervention. Second, charter school networks already implementing the policies 
and practices associated with success could expand, opening up new schools using the same 
policies and practices. After the network had reached a particular size or scale, to the point at 
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which the context in which their schools were operating had changed, the impacts of the newly 
opened schools could be compared with previous estimates of the impacts of network schools 
operating when the network was smaller. If the newly opened schools were equally successful, 
this would be evidence that the network’s policies and practices remained successful in this new 
context. 12 

Fryer (2014) provides an excellent example of the first approach to learning more about 
charter school policies and practices. The author identified a bundle of five policies and practices 
found by Dobbie and Fryer (2013) to be consistently present in successful New York City 
charter schools and implemented these practices in a set of traditional public schools in Houston, 
Texas. These practices included increased instructional time, strategies to increase human capital 
of teachers and administrators, high-dosage tutoring, frequent use of data to infonn instruction, 
and a culture of high expectations. In one set of schools, random assignment was used to 
detennine which schools would implement the practices. In a second set of schools, non- 
experimental methods were used to detennine the effect of the practices on student outcomes. 

Findings from Fryer (2014) give us reason to be optimistic that research on the factors 
associated with successful charter schools can identify policies and practices that can be used 
successfully in other schools — in this case, traditional public schools. The implementation of the 
bundle of charter school policies and practices in Houston led to an annual increase in student 
achievement of 0.206 standard deviations in math and 0.043 standard deviations in reading. The 
intervention also led to a modest increase in attendance at the secondary school level, although 
the impact at the elementary school level was not statistically significant. Fryer (2014) also 
reported that a similar set of policies and practices implemented in traditional public schools in 
Denver, Colorado had similar positive effects (based on a non-experimental design). 

We know less about the extent to which successful charter school networks maintain 
positive impacts as they grow and reach a scale at which they could struggle to replicate their 
practices effectively. A key reason for this lack of evidence is that most charter school networks 
are relatively small, especially in terms of their presence in a single city or district. KIPP is 
currently the largest network, with 161 KIPP schools slated to be open in 2014-2015, and is only 
beginning to reach the point where there are a large number of KIPP schools in any given 
district. 13 

At this point, we must rely on suggestive evidence about whether successful charter school 
networks can maintain positive impacts as they grow. CREDO (2013c) found that the early 
perfonnance of CMOs is a good predictor of their later performance. CMOs typically grow over 
time, suggesting that at least in the early stages of CMO growth, opening new schools does not 
water down the practices that made them successful. In the case of KIPP, Tuttle et al. (2013) 
conducted an analysis that compared impacts in a set of 4 1 established KIPP schools based on 
data from 2001-2002 to 2010-2011 with impacts in a set of 5 newly opened KIPP schools based 
on data from 2009-2010 to 2010-2011. Math impacts after one year of attendance in the newly 
opened schools were slightly less positive than those in the established schools, though impacts 


12 

“ It would still be true, however, that we would not know for sure exactly which of the network’s policies or 
practices was responsible for its success. 

1 3 

The largest KIPP region, Houston, current has 24 schools. 
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were positive and significant in both cases (0.12 versus 0.15 standard deviations). In reading, 
impacts in the newly opened schools were more positive than those in the established schools 
and again positive and significant in both cases (0.12 versus 0.05). So based on this very limited 
evidence, we do not see signs that impacts are diminishing as charter school networks scale up. 
However, even the largest networks have scaled up only modestly thus far. 14 

Overall, learning more about the policies and practices likely to lead to success among 
charter schools, and potentially among traditional public schools, requires continuing research in 
three areas. First, researchers should continue conducting research in which rigorous methods are 
used to estimate charter school impacts and these impacts are correlated with school policies, 
practices, and other relevant factors. This research will generate hypotheses about what factors 
actually lead to school success. Second, to the extent possible, researchers should work with 
policymakers to purposefully implement these factors in other schools and investigate their 
effects in those settings. Third, to the extent that charter school networks that have been found to 
have positive effects expand, researchers should examine whether schools in the network remain 
as effective as the size of the network increases, especially within individual districts. 


14 However, the KIPP Foundation received a 2010 Investing in Innovation (i3) grant from the U.S. Department of 
Education, designed to promote the expansion of the KIPP network. Mathematica Policy Research is currently 
conducting an evaluation of KIPP’s scale-up activities, including estimating impacts among newly opened KIPP 
schools. 
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